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Introduction 


As schools and classrooms explore technology-based, student-centered, personalized approaches 
to teaching and learning, their efforts to innovate can be hamstrung by archaic district operating 
systems that do not allow them to take advantage of new technologies, to work with smaller 
startup companies, or to quickly make and implement decisions. When he was at the helm of the 
New York City Department of Education (NYCDOE), Chancellor Joel Klein attempted to remedy 
this problem by creating the iZone, an office explicitly charged with fostering innovation from 
within the school district. From 2012 to 2014, 1 designed and served as executive director of 
Innovate NYC Schools, the iZone project that focused on new structures for district procurement 
and decision making. As the first urban school system to make design thinking a routine part of 
its practice with educators, we created alternatives to established procurement processes and 
built collaborations between the school district and early-stage education technology companies. 
We also provided regular support to entrepreneurs, investors, and innovators who sought to do 
business with NYCDOE. Using the philosophy and examples from NYCDOE, this paper lays out 
steps school systems can take to foster innovative practices throughout the organization and 
develop new instructional tools and practices through nimble procurement procedures. 

A Brief History of the NYCDOE iZone 

In 2002, Michael Bloomberg became the first mayor of a large city to be granted effective control 
of his schools. He appointed Joel Klein as chancellor, who, over the next eight years, proceeded to 
make changes of unprecedented depth, breadth, and controversy in the structure, governance, and 
operations of both schools and the NYCDOE central office. These changes were carried on by his 
successors, Cathie Black and Dennis Walcott, until the 2014 election of Mayor Bill de Blasio. 

Among these changes was the creation in 2010 of an Office of Innovation, also known as the 
“iZone,” to pursue approaches to personalized learning that were new, unfamiliar, or antithetical 
to the prevailing habits and practices within the district. The iZone was to be the research and 
development lab for the district at large, developing and testing new technologies, approaches 
to teaching and learning, and resource allocations that, if proved promising, could then be rolled 
out more broadly. iZone projects were initially funded from the city’s $400 million portion of New 
York State’s Race to the Top grant, augmented later with Investing in Innovation grants, foundation 
support administered through the Fund for Public Schools, and ordinary tax levy monies. 

As with NYCDOE’s precedent Autonomy and Empowerment Zones, participation in the iZone 
was a voluntary decision by principals and teachers. In exchange for certain benefits (access to 
new instructional tools, encouragement and support in reimagining many of the conventional 
inputs to school structure, release time and money for professional development, participation 
in professional learning communities, etc.), iZone schools agreed to design and adopt new 
instructional practices and to participate in research to monitor their impacts. 
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The iZone Philosophy 

The goal of the iZone was to enable effective personalized teaching and learning. Over time, its 

theory of action came to depend on three interlinked assumptions: 

1. If you want teachers to instruct differently, you must be in the classroom showing them 
how. 

2. In order for teachers to have the freedom, support, and incentives to instruct differently, 
you must be active in reshaping policy at the district, state, and national levels. 

3. Since everything schools do is enabled by things they buy, you must be active in the 
marketplace, functioning as “smart demand” to help vendors build better products for the 
district to procure in better ways. 



iZONE 360 

Cohort re-thinking basic 
assumptions of school structure 
and design 


A 


iZONE 

launched in 2010 with 81 schools; 
grew to 400 


T 


INNOVATE NYC 
(launched 2012) 

Procurement as problem solving 


Obtaining and optimizing for new technologies: The iZone launched in 2010 with a strong focus 
on obtaining the tools of instructional technology. Laptops, iPads, learning management systems 
platforms and the like were seen as innovative. At launch, therefore, a central role of the iZone 
was to prepare and support schools in these efforts— negotiating vendor contracts, coordinating 
network upgrades, and providing schools with laptops and tablets. 

The iZone’s debut effort was iLearn, a blended learning platform populated with content from a 
dozen publishers and supported by extensive training and professional development provided by 
iZone staff. Eighty-one schools joined iLearn in its first year; eventually iLearn grew to more than 
300 middle and high schools whose intensity of use varied from occasional online credit recovery 
to full-blown blended learning and flipped-classroom models. 1 

Creating new school models and new regulatory mindsets: But some schools wanted to go 
beyond simply providing new technologies and focus on whole-school redesign. In parallel with 
iLearn, iZone360 launched to work with a different cohort of schools to design new school models 
in which personalized practices could take root. An initial cohort of 28 middle and high schools 
(ultimately growing to 50) would work with dedicated “innovation coaches” to rethink school 
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inputs like place, space, time, scheduling, and teacher roles. This would be accompanied by a 
massive infrastructure upgrade at those schools to prepare them to intersect with the technology- 
enabled vision of iLearn. 2 

From the outset, regulatory change in the areas of credit-granting, accountability frameworks, 
professional development, and scheduling and staff assignment was an integral objective. For 
example, in addition to the schools participating in iZone 360, another half-dozen iZone schools 
spent a year developing their own competency-based models in which students would progress, 
not as age-based cohorts, but at whatever pace was necessary for them to demonstrate mastery. 
Known as the “asynchronous pilots,” this work in particular required regulatory revamping of 
student accreditation and teacher evaluation regulations. 

The iZone architects recognized that these changes had to be durable options— not idiosyncratic 
exceptions— so that future innovation would have fewer obstacles to overcome. 

Shifting to sm The third programmatic leg of the iZone, coming more than a year 

after the launch of iLearn and iZone360, was focused on the marketplace. Known as “Innovate 
NYC Schools” (henceforth, “Innovate”), this program was based on the often-ignored reality that 
everything that schools do is purchased in one way or another. Still, districts are terrible customers, 
and the process, culture, and incentive structures of school and district purchasing drive away 
the most potentially high-value vendors. Those vendors that do participate are rewarded not 
for creativity or collaboration but for their compliance with processes and cultural norms that 
are not in any way optimized for high-quality outputs. The goal of Innovate’s marketplace work 
was to create examples of how New York City (and by implication, other districts) could change 
its behaviors in ways that would free vendors and other problem solvers to do better work. The 
frameworks employed by Innovate for its projects— User-Centered Design and Lean Startup 
methodology— were themselves borrowed from the startup community. 3 They would initiate 
a shift in the way the iZone thought about its work, from innovation as the adoption of new 
tools, to innovation as a set of habits and processes for iterative problem solving in any domain. 

An essential component of the work was finding novel ways to bring educators and innovative 
providers together. 

Innovate NYC Schools 

BRIDGING THE GAP WITH THE STARTUP COMMUNITY 

While the rest of the iZone staff, including the iLearn and iZone360 teams, was based in district 
offices, we began the Innovate project by locating the team in a public co-working space that 
housed startups. It was important for both practical and symbolic reasons that Innovate be 
embedded in the startup community, easily accessible to potential collaborators without their 
having to present identification and clear security. The fact that we were coming to them rather 
than having them come to us was also a way of signaling our respect for the value of their time; 
absence of respect was one of the most common complaints of NYCDOE vendors. Crucially, 
locating outside of the central administration was also a kind of insurance policy: innovation 
skunk-works inside large organizations seldom succeed, mostly due to the difficulty of escaping 
the gravitational pull of the surrounding culture. Those that do succeed are almost always located 
separately from the parent organization because they are then able to develop their own norms 
and practices. The effort to relocate was an important first step, which carried as much weight as 
subsequent Innovate projects, and helped shape public opinion about the work. 
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In the case of Innovate’s relocation, the NYCDOE had never before rented co-working space and 
their legal department was unsure as to whether and how it could be done. We recruited the 
support of two deputy chancellors, whose pressure on the legal department not only enabled us to 
execute the co-working membership agreement but also sent an important signal that novel and 
unconventional ways of operating were possible and would be supported. 

THE VALUE OF INFORMAL CHANNELS 

Of the many frustrations experienced by startups when dealing with the central DOE, perhaps the 
most maddening and most characteristic was the inability to get timely, actionable answers to 
routine questions. At the same time, most principals and central administrators had never had an 
informal conversation with an employee from a startup (as opposed to a sales pitch). An important 
goal of Innovate NYC Schools was to bridge this gap. In our first year we ran a number of trials of 
different formats to see which were most productive, best liked, and easiest to administer. 


BRIDGING THE COMMUNICATIONS GAP 

Monthly Office Hours: Anyone (though mostly startups) could book time with an Innovate 
staffer to talk or ask about anything related to education technology or the NYCDOE. 

Reverse Field Trips: We rented buses, packed lunches, and bundled startups off to visit 
schools. There they would observe classes and have a series of open-ended conversations 
with principals, teachers, and tech coordinators about the day-to-day challenges they 
faced. Although they could describe their product in response to questions from the 
educators, the startups were barred from doing product demos or pitches: the point was 
for them to learn, not to sell. 

Fireside Chats: We held informal 90-minute afternoon sessions with no set agenda at 
our co-working space, hosting school and central office staff to talk about their work and 
answer questions to a broad audience. 

Shark Tanks: Our most popular effort. Education technology developers could receive 
direct feedback on their products and sales pitches from teachers, administrators, parents, 
and students. These evening events, held at co-working spaces or other neutral ground, were 
typically organized by theme: for example, formative assessment dashboards, classroom 
management tools, or assistive technology for special-needs students. Each startup would 
have the chance to demonstrate their product to a panel reflecting their target audience, 
who would then provide constructive feedback on its perceived strengths and weaknesses. 
While the field trips and fireside chats appealed to startups’ general curiosity and need for 
context, the shark tanks provided undiluted doses of what they most wanted: concentrated 
feedback from potential customers on product/market fit, which supported the creation of 
more relevant and effective education technology tools for the classroom. 


REFRAMING PROCUREMENT AS PROBLEM SOLVING 

The goals of refining the district’s procurement process went beyond creating the ability to buy 
better things on better terms— they represented a shift in mindset. Whereas before, procurement 
signified a series of bidding and contracting procedures to comply with and used to minimize risk, 
Innovate’s approach to procurement focused more directly on the needs of the end users. 
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New York, like all large districts, offered numerous examples of deeply flawed frameworks at 
every stage of procurement: problem definition, solution sourcing, selection, and implementation 
and evaluation . 4 When the Innovate work began, teachers, principals, parents, and students— the 
intended users of much of what is acquired— had a negligible role in defining any of the processes 
through which decisions were made. For vendors, having a product that pleased the end user 
mattered much less than being able to persevere through an opaque process of indeterminate 
length and uncertain outcome, a trait which tilted the odds in favor of large established companies 
whose products educators were frequently trying to escape from, not toward . 5 

The disconnect between these conditions and the proliferation of useful, well-designed, inexpensive 
education technology was particularly apparent in New York, which had become a powerful center of 
edtech entrepreneurship. Many of these young companies were eager to connect with educators and 
administrators, not just for sales opportunities, but to create collaborative relationships that would 
result in better products built on a better understanding of the day-to-day realities of schools. 


Traditional Procurement Innovative Procurement 


Experts identify and define problem. 

Problem is defined by those who will use solutions. 

Problem is framed as a specification to be met. 

Problem is broadly defined to encourage large and 
diverse community of problem solvers. 

Incentives to participate are either compliance 
with a mandate or financial. 

Incentives to participate are the development of new 
skills, relationship building, and participation in a 
rewarding experience. 

Experts winnow the submissions, the “best” 
are implemented at scale, as-is, across many 
schools. One large, high-stakes decision. 

Practitioners choose those that will be implemented 
and refined in their classrooms. Many small, low-stakes 
decisions. 


Over the course of two years, Innovate hosted a series of procurement initiatives. They were similar 
in that they relied on field research to identify the problem to be solved and empowered the end 
user to have a say in the solution that was developed. What differed was the length of time and 
means of procuring the solutions. 
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THE GAP APP CHALLENGE: 

AN OPEN CALL TO SOFTWARE DEVELOPERS 

The Federal Investing in Innovation grant that funded Innovate’s work was focused on 
identifying the key problems of middle school math learning. Finding solutions to these 
problems would become the object of an open call to software developers, with the best 
products identified and then pushed out to hundreds of schools. 

A search of the math deficit literature returned no useful conclusions as to the nature of the 
problem. 6 Working with a design firm, we spent a month speaking with middle school math 
teachers, instructional coaches, principals, and students across the city about what they 
perceived to be the problems with middle school math. 

What emerged was that, from an instructional perspective, the problem was structural. 
Teachers and coaches grappled with a broad range of mathematical understanding and 
ability within a single middle school classroom, making effective personalized instruction 
nearly impossible. Teachers wanted tools to help them manage and bridge that gap within 
the typical urban classroom. 

For the five month-long "GapApp Challenge,” educators and edtech startups were the 
participants as well the audience. Their motivations for participating were symmetric: for 
teachers, to be more effective as instructors; for the startups, to sell more of their products. 
With that in mind, we set up the Challenge to embody teacher voice at each significant step. 

1. Teachers' needs were distilled and transmitted via the problem definition. 

2. Startups were given numerous informal opportunities to interact with teachers, 
principals, and central office staff to ask questions and absorb the cultural and 
practice norms. 

3. The key factor in selecting winners was teachers’ evaluations of the submitted apps. 

We also added a uniquely valuable incentive for the startups: any company entering the 
Challenge— regardless of whether they advanced to the finals— was eligible to be selected 
by an iZone school to be in residence on their campus, working with teachers to refine their 
product to meet the needs of real urban classrooms. Although there were also financial 
incentives, our surveys showed the most powerful motivation for both the teachers and the 
startups was the opportunity for that deep collaboration. 7 


Since the GapApp Challenge was the first attempt at this type of matchmaking and collaboration, 
our primary goals were visibility, widespread participation, and delight in the process. We wanted 
to remain deliberately open-minded about what sort of software we were looking for to encourage 
creativity. We anticipated a few dozen submissions and ended up with almost 200. The Challenge 
was widely covered in print and social media. The test beds established between iZone schools and 
edtech startups became models for national grant programs from major foundations and the US 
Department of Education. 8 Most importantly, surveys of the educators and startups involved were 
highly favorable, with Net Promoter scores of 87. 9 
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MUSIC EDUCATION HACKATHON: WEEKEND CROWD-SOURCING 

We developed a second crowd-sourcing matchmaking event that took place over a 
weekend with the goal of establishing a generalizable toolkit— a set of habits and 
attitudes that could be pointed at any problem. User research for GapApp took one 
month; for this challenge, it was conducted during one evening. Like the math teachers in 
GapApp, the music educators sought tools to tailor instruction for students with a wide 
range of abilities. 

Like GappApp, the goal of this challenge was to channel the voice of educators into a 
provocation that problem solvers could build for, and then close the loop by having those 
educators choose products for their classrooms or teams to collaborate with for further 
development. Though we partnered with a high-level sponsor (Spotify), there were no 
prizes for participation and any follow-on collaboration would have to be organized 
among the participants. As with GapApp, participation and satisfaction levels were high. 
NYCDOE arts educators who had felt overlooked and underfunded by the reforms of 
the prior few years were pleased to be the subject of an unusual amount of attention 
and work. We demonstrated that we could quickly negotiate and deliver a low-stress 
partnership with a well-known mass-market brand without compromising the dignity or 
integrity of the NYCDOE. Most importantly, the success of MusicEdHack demonstrated 
that the GapApp was not a fluke. It enabled the credibility that comes with a track record, 
and gave us momentum. 


SEEKING CENTRAL OFFICE BUY-IN 

We wanted to establish the utility of our problem-solving approach to central district workers by 
developing a program that would directly benefit them and their constituents— the families whose 
children were enrolled in the school district. 

New York City has the most extensive program of high school choice in the country: with rare 
exceptions every middle school student must actively select her high school. This high-stakes 
decision was made more stressful and complex by the proliferation of new themed schools during 
Joel Klein’s administration, such that families had to select from more than 700 program choices. 
Students rank 12 choices in order of preference and then are algorithmically matched in a process 
that consistently provides 70 percent of students with one of their top three choices. Despite the 
favorable outcomes, the process confounded parents, students, principals, and guidance counselors. 

We began as before, with user research, spending several weeks in schools and homes interviewing 
middle school students, families, principals, and guidance counselors, as well as 9th grade students 
and families who had been through the complete admissions process. What emerged was 
frustration with the dearth of tools available to provide information, guidance, and management 
for this daunting process. Each year, the Office of Student Enrollment (OSE) staff laboriously 
produced and distributed over 100,000 copies of the “Handbook,” a two-inch thick compendium of 
information about each school that cost more than $6 per copy for printing alone. They held school 
fairs in each borough where high school program representatives gathered at tables in cafeterias 
and gyms to answer questions from prospective students and parents. Middle school guidance 
counselors received updates each year on any changes to the process and instructions on how to 
advise students in filling out their machine-readable preference sheets. 
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Yet there were no tools to help anyone manage this daunting process, nothing like what we have 
come to expect in daily life for even routine search-and-decision— like movie search— let alone 
for high-stakes decisions. Many of our families didn’t speak English at home, had never had to 
navigate a choice process like this, and so could do little to help their children make one of the 
most consequential decisions of their lives thus far. School guidance staff was simply overwhelmed 
and lacked the tools to help students or even manage their own processes in a reasonable way. Our 
field research revealed this to be a major factor in the opacity and stress people experienced when 
dealing with high school choice, and it was something we hoped that Innovate could help to improve. 


THE SCHOOL CHOICE DESIGN CHARETTE: 

PARTICIPATION BY INVITATION 

Because the New York City high school admissions process would require particular 
solutions— not applicable to other markets— instead of an open software challenge we 
held an invitation-only school choice design charette (SCDC). We reached out to builders 
of decision-support tools for other areas of overabundant choice, including consumer 
products, college and graduate school, job search, and dating sites. Six were selected to 
participate in a two-month charette in which they would have access to our research and 
panels of parents, students, and counselors for testing and feedback, and sessions with 
OSE employees to explain the rules, logic, and constraints behind the admissions process. 
Rather than compete for a prize, participants would receive a $12,000 stipend for their 
participation, contingent only on producing a useful free tool, maintained for at least one 
year. The tool had to draw from official NYCDOE school data (it could also use other data 
sources but must include ours) and not inappropriately collect student information. 

Again, our core objective was to change the perspective and behaviors of the institutional 
actors; in this case, getting OSE to attend to the experience of families participating in 
school choice by creating simple, open public access to data and paying companies for 
participating in a process rather than delivering a product. One of the SCDC provisions 
was that, since the developers would make their applications freely available, OSE would 
publicly promote their availability. This might seem uncontroversial given that the project 
was designed to provide families with tools to support OSE’s process, but it represented 
a major change from a system where information and process were tightly controlled and 
specified at each step of the way. 

The charette resulted in the development of six different online school information tools for 
families to work with. Students were invited to vote on the app they found most useful, and 
all six were made available to families through the DOE’s website. The open data was also 
maintained for 18 months so that other developers (as well as policy analysts and other 
stakeholders) could make use of it. 


NAVIGATING POLITICAL CHANGE 

After the 2014 mayoral election and the announcement by Chancellor Walcott that he would be 
stepping down, we decided to host a “Chancellor’s Challenge,” in the hopes of institutionalizing our 
problem-solving frameworks so they could carry over to a new administration. 
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THE CHANCELLOR’S CHALLENGE: 

INNOVATIVE AND TRADITIONAL PROCUREMENT 

We invited any central DOE employee to submit an intractable problem, an idea that seemed 
impractically blue-sky, or a project that had been previously rejected as too high risk. In turn 
we required that the winning office commit two dedicated staff members for one day each 
week and that the deputy chancellor responsible for the office be available to meet with us as 
necessary. 

We received applications from about a dozen offices on topics ranging from school lunch 
participation to facilities maintenance. We chose one that was particularly knotty, relating to 
transportation for special education students. Those services are provided by two separate 
offices working jointly: the offices of Special Education and of Pupil Transportation, which, 
though they report to the same deputy chancellor, have very different practice cultures and 
management styles. 

This project was a hybrid of our innovative procurement process and the bureaucracy’s 
traditional decision-making framework. The research phase would lead to a set of ideas for 
potential prototypes, the deputy chancellor would select which of those ideas to pursue in the 
prototyping phase, and then, the prototypes would be tested and refined in small-scale trials. 

For the field research, we took Special Education and Pupil Transportation staff to interview 
students and their parents, teachers, and principals. We brought them on early morning 
depot visits to chat with bus drivers. We rode the buses and shadowed assistant principals 
at dismissal time, and dove into the legal and practical arcana that define special education 
busing. Not surprisingly, we gleaned a great deal of local expertise and wisdom very close 
to the ground that had never been part of any systemic service feedback loops. Existing 
processes for bus scheduling and routing, for example, did not draw on the day-to-day 
experiences of bus operators or families and so were much less efficient than they otherwise 
could be. As with the SCDC problem definition, these kinds of information inefficiencies 
readily lent themselves to service and tool improvements. 

In December 2013, the final month of the Bloomberg/Wolcott administration, the field 
research was distilled into a report recommending three service prototypes to test. Since 
a key issue for both parents and bus operators was the poor communication of routine 
information related to routes, pickup times, and student needs, these proposals focused on 
that low-hanging fruit— a series of simple, well-designed forms to gather information and 
communicate it in a timely manner to those who needed it. While we could have suggested 
instead an expensive, technology-intensive approach to these problems, this inexpensive, low- 
tech approach lent itself much more readily to the quick experimentation and refinement that 
would lead to a more robust and useful solution. 

However, the following months were marked by significant turnover at the middle and 
upper levels of the DOE corresponding to the change in administration, its priorities, and its 
attitude toward work begun under its predecessors. It took four months for the new divisional 
leadership to authorize the design of two prototypes. Those were completed in just two 
months and presentations made to the divisional leadership outlining how to proceed to 
testing and deployment. However, no further meetings took place, and the prototypes were 
never implemented. There was never a definitive communication that the work would not 
proceed. Rather, as is often the case in bureaucratic organizations, it was simply left to die 
from lack of active support. 
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The Evolution of the iZone 


By September 2014, less than a year into the de Blasio/Farina administration, 80 percent of 
iZone senior leadership had departed. iZone360— the original whole-school redesign program— 
which had later also taken up the framework of User-Centered Design/Lean Startup Challenges and 
pioneered it as a uniquely powerful approach to practitioner-driven school reform— was dissolved. 
The remainder of the Innovate NYC Schools team was transferred— along with Hearn— from the 
disbanded Division of Talent, Labor, and Innovation to the IT Division, subsumed within one of the 
offices whose behavior it had sought to change. 

Those aspects of the Innovate marketplace work that were directly connected with validating 
and improving edtech products continued. Funding was secured from the Bill & Melinda 
Gates Foundation to extend the existing Innovate test beds for three more years. The popular 
#SharkTankEDU program would continue as a lightweight, low-overhead process for connecting 
edtech developers to educators and parents for product feedback. 

What would not continue were the arcs of work with arguably the greatest leverage: modeling for 
the central office and for school leaders a new way of defining problems and seeking solutions. In 
essence, Innovate returned to a narrower focus on classroom technology, a focus we had sought to 
go beyond. 

That is not to say the work of those two key years, 2012 and 2013, left no trace. Much of what was 
created there, like the test beds, shark tanks, and school design challenges, became a template for 
work that is continuing in other districts and in cross-regional collaborations. The User-Centered 
Design and Lean Startup principles, which had never been systematically applied by a district across 
many levels and many projects, are now generally accepted professional development activities 
around the country. Among policymakers, the centrality of procurement, contracting, and IT policy 
that was the focus of the Innovate district-level work, is now widely recognized as a bottleneck to 
the successful implementation of governance and instructional improvements. In New York, too, of 
course, the iZone-introduced blended learning and edtech outreach efforts continue. 

What has been dropped in NYC and not yet picked up elsewhere is the example of a district 
creating ongoing provocations to itself. What was innovative in Year 1 is by definition not innovative 
in Year 3, nor is it “innovative” for the instigators to simply refine, reproduce, or scale that work. 
Those are worthwhile activities, to be sure, and the originators of that work have a role to play in 
helping others spread those activities. A district edtech office that ran such programs would be 
unusually forward thinking. 

But that cannot be the primary work of an “Office of Innovation.” To make it so is to critically 
undermine the ability of that group to act in an authentically innovative manner. Once you are 
concerned primarily with the replication of successful past projects, once you have more answers 
than questions, once you are comfortable within an organization you have sought to change and 
the organization is comfortable with you, you aren’t “innovating” anymore. 
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Endnotes 


1. Another example of the iZone’s emphasis on the transformative possibilities of software was the School of One 
program, in which math students received a personalized computer-generated “playlist” detailing the day’s learning 
activities, based on their mastery of concepts. 

2. It later turned out that many principals were far more motivated by the chance to get free laptops than they were 
by the promise of whole-school transformation. 

3. In a nutshell, User-Centered Design is “a framework of processes ... in which the needs, wants, and limitations of 
end users of a product, service, or process are given extensive attention at each stage of the design process.” Lean 
Startup methodology advocates investing only small amounts of time and money in each of the early stages of design 
for a new product or service. This allows assumptions and decisions to be tested with users early and often, and those 
changes to be incorporated into the next rapid iteration. At the very least, these are processes for minimizing the risk 
and the costs of failure, though very different from those that schools have traditionally employed. 

4. The establishment of the Fund for Public Schools, which administered a portion of iZone funding, was itself an 
attempt to route around procurement by having an entity controlled by the district that was yet not required to follow 
its procurement practices. Without this flexibility, much of the iZone work would have been impossible. 

5. See Tricia Maas and Robin Lake, A Blueprint for Effective and Adaptable School District Procurement (Seattle, WA: 
Center on Reinventing Public Education, 2015); Robin Lake and Steven Hodas, “The Procurement Tightrope Shouldn't 
Tie Districts in Knots,” The Lens (blog), Center on Reinventing Public Education, January 6, 2015. 

6. The framing assumption was that the “problem” would be topical or curricular, that if only students could properly 
perform ratios or percentages or equations that the remainder of their math education would proceed more 
successfully. This was the type of evidence the researchers sought. 

7. This was not surprising, since in most hackathons, challenges and open-source collaborations financial incentives 
are minimal to non-existent. The cash prizes served more as a means to capture attention than to drive participation. 

8. In the summer of 2013 the Bill & Melinda Gates Foundation released the first of a series of RFPs to promote 
new models for evaluating the efficacy of edtech products, which came to be known as “Short-Cycle Evaluation 
Challenges” (SCEC). Like the GapApp, the SCEC were designed to facilitate school-based test beds in which 
educators and developers would each benefit from extended collaboration. 

9. The NetPromoter score is a cross-industry standard for gauging user satisfaction, consisting of one question: “How 
likely are you to recommend this to a friend?" Scores above 70 are considered excellent. For example, Apple is the 
only electronic company consistently scoring in the low 70s. 
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